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Evaluation of a Public Tax-Supported Alternative School^ 



Richard J, Rankin and A. J, Gaite 
University of Oregon 



, The paper describes the evaluation of a public tax-supported alternative 

school. The emphasis is upon the problems involved in the evaluation of 
I this kind of school and suggested methods of overcominq them* A post-ore 
design combined with control groups is described, and the pros and cons of 
involvement and detachment on the oart of the evaluators is considered. The 

o 

_ problems associated with reactivity of the program staff and students to the 

o 

evaluators are discussed. The main findings and results are presented. 



Paper read at the annual meeting of the American Educational Research 
Association, Chicago, 1974. 



Evaluation of a Public Tax-Supported Alternative School' 

'Richard J. Rankin and A. J. H. Gaite 
University of Oregon 

The main emphasis in this paper is upon the methods, problems, and 
issues involved in an evaluation program designed to provide information to 
assist a school, board in determining the fate of a publicly supported alterna- 
tive (free) school program designed for 8th, 9th, and 10th graders. This 
was a real evaluation, not a theoretical exercise. The opportunity to 
evaluate a tax-supported alternative (free) school was considered by the 
authors too valuable an opportunity to pass up notwithstanding the fact that 
the school board placed a number of limiting and ofttimes irksome constraints ^ 
upon the exercise. Alternative schools abound in the USA and doubtless some 
are evaluated; however, very very few tif any) tax-supported schools of this 
type have undergone a full scale evaluation. Thus, constraints and diffi- 
culties notwithstanding the opportunity were extremely inviting. 

The major parameters set before the evaluators by the school board were: 
(1) The need for academic data; (2) The need for attitude data from students, 
parents, and school personnel; (3) The need for a control group; and (4) The _ 
requirement that the study be finished in 10 months, bridging two school years. 

The problems generated by these restrictions were compounded by the 
self-selected nature of the student body of the tree school. The staff of the 
school tended to be very cooperative; however, this cooperation led to 
problems of reactivity and the problem of evaluating a program in flux. 

Vaper read at the annual meeting, of the American Educational Research 
._ Associ a4:ix)n^_Chi cago ,_lS7-4^ ~.~ 



Theoretical Framework 

The evaluation was based upon a threefold structure: (1) Description of 
the Drogram; (2) Description of the students; (3) Comparison of the students 
with the most comparable control group. 

The evaluation was by necessity a quasi -experimental design with all 
statistics chosen to Illustrate most clearly the characteristics of students 
or direction of change taken by a particular group. The problems Inherent 
in evaluating a relatively new and ongoing program (ongoing at least as far 
as the staff was concerned) were such that the whole exercise was necessarily 
a combination of the formative and summative aspects of evaluation. The 
dictates of the school board placed the emphasis upon the summative procedures 

The need to complete the evaluation over a ten (10) fhonth period 
spanning two (2) school years (March-December) dictated the overall design 
selected. Ideally, a matched sample control group might have been selected^ 
though necessarily they would be different in that such a sample v/ould in 
fact be attending regular schools and not the alternative school. The 
problem of time, so often forgotten in a theoretical exercise! and so of ten 
crucial in a real study> forced the selection of a comparison group drawn 
from a regular school in the same area from which a large nuiriber of those 
attending the alternative school came. Additional data from a comparison 
group were gathered from a school judged to be most similar to the type of 
school from which the free school students came. Students in this situation 
were given one part of the Stanford Achievement Test and the attitude test. 
This was done to satisfy a time limit for data gathering Imposed by 
officials in the control school. File data provided the ability measures. 
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The second part of the design utilized a DOst>pre design wherein 
students newly come to the alternative school in the fall (September) were 
Gvaluated/measured/observed and compared with students who the previous 
sumner (June) had been In the program for a year* While not Ideal , this 
procedure was again dictated by the demands of time but nevertheless did 
permit some assessment of "growth" or "change" which could be attributed 
to the alternative school program. Necessarily such a procedure assumes 
comparability of the two alternative school groups and depended upon there 
being no change in the admission procedures, etc., of the school. This 
criteria was, as far as can be assessed, met. 

Methods 

At the start of the evaluation, a decision had to be made regarding 
the overall evaluation method that was to be usedri Were the ev9lii^^ 
to be detached? Were they to be involved participant observers? In this 
Instance, the^evalua that little evaluation could in fact 

take place unless the evaluators were accepted by students and staff ^^^^ 
general part of the scenery and structure of the school. In short, 
detachment, though a laudlble ideal, did not seem to be a real posslMllty. 
At the same time there seemed little point in pretending that an eyaiuator 
was not in the last resort a person ma kInjgL. judgments about staff, students, 
and the whole school. In these circumstances, it seemed unlikely that the 
evaluators could^ie totally accepted. Familiarity and ordinariness could 
and vias accomplished, however, by spending hundreds of hours In the school 
(over forty (40) visits were made by the ievaluators in less than two- i^chpoT 
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terms) such that the presence of evaluators talking, looking, and perhaps 
testing, became just a part of the general scene. 

Thus the description of the program was not based only upon the stated 
objectives of the group, but was basically an observation of the group In 
action. Attendance data was gathered from file records kept by the 
personnel and this form of data was supplemented by direct observation of 
students in the building and In the classrooms. The alternative school 
gr^up was tested with the Stanford Achievement Test, the Brown Holtzman 
Survey of Study Habits and Attitudes, and thlT^Short Form Academic Aptitude 
Test. A test designed to measure attitude toward their specific school was 
constructed and administered. Further evaluation was based upon the grades 
assinned to the students by the teachers, even though this student evaluation 
was relatively free form. Records from permanent files were useful in 
evaluating projected achievement^scbres from early school attendance. 
Discussion with students was systematically pursued and observations of work 
habits were gathered. The expectations of the staff were analyzed In 
comparison with what could be expected knowing the characteristics of the 
students. 

Data relating to ppi#jons of the members of the community and of other 
school officials were contrasted with objective evidence available. 
Incidental or non-relative data were gathered and were contrasted with what 
was expected by other school administrators. 

The intent In the study was that the data gathered be direct and primary 
wherever possible, and this aim was accomplished. Intelligence and 
achievement tests were administered even though other tests may have been v 
vai-l-abl^-in t-he-fi4e^ — Thi-s-^as--done-^beeaijseH:4teHresi5onses-^f--the-^ — 



to thn tests were in themselves data, and because more complete data was 
thus available. ^ 

Data* from the control group (students In a reqular junior high school) 
was in part gathered by the evaluators (Stanford Achievement Test, attitude 
test, interviews regarding oart-tlme jobs, and attitudes) and In part 
obtained from the students' permanent record files (Intelligence test scores, 
achievement tests). 

i 

Resul ts ' 

C haracteristics of the Students . The student body was predominantly 
male, almost in the ratio of 2/1. There was a slight 6 point difference . 
in favor of the females in the I.Q. of the students-rWho were- l4i the program 
for at least the year. In this alternative school, the students aDproxImate 
the conventional definition of normal intelligence with a combined across 
groups intelligence of approximately 90.* The variances were as expected, 
approximately 15 I.Q. ooints. Though the alternative school group had a low 
normal I.Q., their reported self perception was that of dull. This 
perception may have been reinforced by the control groupi5,mean I.Q. of 110 
again with a variance of 15. Thus in the context of the school district the 
alternative school group should not be contrasted with a normal population 
with a mean of 100, but with a group having above average intelligence. 
Teachers in the regular schools may perceive an I.Q. of 90 as dull when 
compared to the control group figure of 110. 

About t^o-thirds of the students in the alternative school were from 
single-parent homes contrasted with one-third from such homes In the control 
group. 



The alternative school students tend to like their school , compared 
to those in the control group who were more negiative. This was found both 
in interview data and in a short attitude*tO'-schoo1 scale. , - 

Achievement . Both the finishing class and the entering class In t\te/..-.r-M 
alternative school have a level of academic achievement that can be ^ ; | 

predicted from- their ability scores. The special treatment of the alternative J 
school does not break this pervasive correlation. Specifically, the 
alternative school students have reading ability at the 6th grade level J 
rather than-'at the normal age/grade 9/10 level. The control group had-4^d44rig n^ 
ability at the 10th grade level. In math computation the alternative school /; 

a . ■ ■ . ^ : ..•-■■)-]••< 

Students were three years behind grade level while the control students.were ; ; 
two years behind grade level. In the math application area, the alternative V 
school students were two years behind grade level while the control school 
was about three months behind grade level. 

In no case was there a noticeable Improvement in the alternative school i:; 
students in academic achievement when entering students were compared with ; . j^; 
finishing students. It was not possible to compare the growth of alternative -{^J; 
school students with similar students in regular schools because there seemed 
no practical way of finding such a group. 

Attitude Change . Student attitude was assessed through Intierviews and. y.i 

' ^: ^ . ■ ■ / /;::.-:v.:-# 

with a standardized school attitude survey: The Brown Hoi tzman, Survey of ''o 

study Habits and Attitudes. This was supplemented Vr14;h a specially cortstructe#i 

a^t1tude-to-school scale. Additional hard data relating -to attitude cfwngife^^^^ 

was gathered by looking at a ttehdiriM^^^ '''^ '■ 'vW 

' On th? Brown Hoi tzrwn there was more than one standard deviation imprbyer-J^ 

n ie nl iM atlikde r U >e d ltOTiatlve SChOftl moved from a imosx three v ..??:^ 



standard deviations below the national mean on total attitude to school to 
slightly less than on# standard deviation below the mean In a period of less 
than a year. In general, those students who go through one year In the 
alternative program finish the year with an attitude to their school almost 
at the same level that the control students have to their school. The control 
students aoproximate the national average. This is a remarkable improvement 
and is confirmed by every piece of data, including interviews with alternative 
school teachers, parents, and the students themselves. If the goal of the 
alternative program was to be attitude change only» then that goal was 
surely met* ■ 

Parents . The Interview and survey data from the parents and guardians of 
the alternative school students Indicated that they were very satisfied with 
the school. This satisfaction was not so much in terms of what the school, 
was doing academically, rather was a response to the fact that: someone was 
trying to help their children. Parents seemed united on the view that the 
school was doing a better job than' the regular school for their children. - 
There v/ere, of course, a 'few caveats but they were far outweighed by those 
satisfied'. This good parental attitude to the school may have been in part due 
to the fact that parents see the school as a last chance for their children, . 
one that otherwise would not be provided. The parents of graduates from the 
school v/ere the most pleased with the operation. This could have been 
properly influenced by the finding that 70 per cent of the graduates were 
either attending school (52 per cent) beyond the alternative school or at 
work (17 per cent). There was a very high percentage of return of the survey 
forms sent to parents and almost two-thirds of the forms contained a page or 



more of additional conment beyond the questions asked directly. This 
certainly reflects a high level of parental Interest in the program. 

Teacher - School Admini strator I ntervl ews and Survey . The regular schooV 
teachers at the junior high and high school levels ivere far less enthusiastic, 
about the alternative school operation than were the parents of al^rnatlve 
school students. Two separate surveys indicated that a lanje number of the 
teachers thought that the financial investment in the alternative school ' 
would be better used reducing class loads in the regular school There was 
some feeling that the school was doing more harm than good. The most solid 
findinq from the teacher surveys was that regular teachers know little about 
the school and the more they knew, the less they liked 1t(l) 

Administrators were more favorably disposed toward the school bjit seemed; 
jealous of its financial support. Extensive interviews most strongly : 
sugqested that some of the administrators thought that a major, use fqr the 
alternative school was as a place where problem students icbuld be sent or 
placed. This was a mismatch with the goals that the. personnel at the 
alternative school had adopted, f of they thought of thCTrielves as' working 
wi th students not disturbed or probi ems but more al iena ted from the world 
than anything else. k 

The most supportive professional group were the regul^ school cpunseliors 
who also had the greatest contact with those students showing potential need 
of the alternative school's services. All district personnel , with the 
exception of those school , appeared 

notably ignorant about the atternative prbgrsrn. 



9 

Discussion 

The school board gave the evaluators four charges which will be used 
as framework for brief discussion. 
( 1 j The Need for Academic Data 

Testing in an alternative school was not as difficult as we had 
first supposed it might be, because the students seemed motivated to 
cooperate. Tlie high compliance with testing regimentation is perhaps 
evidence of the program's success in changing student attitude toward 
some common school activities. 

Academic data from student files was useful in that those students 
who had permanent record files showed similar scores in our own testing 
program. This provides some check on the validity of the evaluators' 
procedures. 

Evaluators could not depend upon the existence of good file data 
because of the transient nature of the population which leaves far too 
many data holes. 

(2) Attltudinal Data Relating to the Students , Parents and School Personnel 
was Requested 

The evaluators were surprised at the power of a conventional test 
in picking up attltudinal change. The success of the instrument may^_ 
be in part due to the fact that we were working with very low/poor 
attitudes, i.e., students who were not topping out on the test. The 
cooperation of parents was Impressive. We think that the parents of 
alternative school children are a valuable source of information which 
is often untapped. Evaluators need not assume that parents of such 
exceptional-students •arenfion-resTJOnsiYe';'^"^ — " 
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(3) Heed for a Control Group 

We could not and did not provide for a conventional control 
group. We did provide a comparison group in a normal school. There 
v/as no way to randomly assign students to treatments; Instead, post* 
pre alternative groups were constructed and t^ese were compared with 
each other and with students in a normal school setting. Adequate 
control groups still need to be built to make more definitive statements 
about the effectiveness of alternative programs. 

(4) The Study was Finished in Ten Months ■ 

4 

This would have been difficult if one of the Investigators had 
not made It a sabbatical project; to say the least there is a tendency 
to underestimate the amount of time each step in evaluation takes. The 
alternative school setting seems to require that in the early stages 
evaluators proceed very slowly to allow s^dents and staff to become 
■ to them. 

In addition to the findings reported here, certain observations . 

^ " '^V-' 

were made by the Investigators that may be Instructive to others attempting^l 

such an operation. f 

(a) The staff ^Tthe alternative school tended to underrate the ability 
and willingness of the students at the alternative school to take 
tests, and it was the staff more than the students who haOo be 
sold on this phase of the evaluation. 

(b) The staff reacted to our presence by starting a more or less 
systematic look at their own product. They attempted to Institute 
changes in entrance requirements that would have made the pre-test 

— <group--^not^xaGtl:y--6omparable— to--^the--post--testr^gro — ~ — ~ 
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(c) The local professional teachers' organization instituted an 
evaluation of Its own, with attendant confusion. 

(d) There are no available control groups and the constructed cpni|ar1so^ 
groups were not as motivated as the alternative schobl student^. 
Teachers and administrators In the comparison school s.wei^e 
reluctant to give up time for evaluation. ' 

(e) All phases of the evaluation took three times as long as 
anticipated. - 

(f) Many teachers and parents were very appreciative of the oppprtunlta? 
to be Included In an evaluation. Thls^lndicates: that^iAeire^^ 1^^^^ 
strong Interest In the public J n the evaluation process. 
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